A Reinforcement Learning Architecture that Transfers Knowledge between Skills when Solving Multiple Tasks

When humans learn several skills to solve multiple tasks, they exhibit an extraordinary capacity to transfer knowledge between them. We present here the last enhanced version of a bio-inspired reinforcementlearning modular architecture able to perform skill-to-skill knowledge transfer and called ‘TERL Transfer Expert Reinforcement Learning model’. TERL architecture is based on a reinforcement-learning actorcritic model where both the actor and the critic have a hierarchical structure, inspired by the mixture-of-experts model, formed by a gating network that selects experts specialising in learning the policies or value functions of different tasks. A key feature of TERL is the capacity of its gating networks to accumulate, in parallel, evidence on the capacity of experts to solve the new tasks so as to increase the responsibility for action of the best ones. A second key feature is the use of two different responsibility signals for the experts’ functioning and learning: this allows the training of multiple experts for each task so that some of them can be later recruited to solve new tasks and ∗P. Tommasiono is with the Nanyang Technological University, Robotics Research Centre, Singapore. E-mail: [email protected] †D. Caligiore, M. Mirolli, and G. Baldassarre are with the Laboratory of Computational Embodied Neuroscience (LOCEN), Istituto di Scienze e Tecnologie della Cognizione (ISTC), Consiglio Nazionale delle Ricerce (CNR), Roma, Italy. E-mail: {daniele.caligiore,marco.mirolli,gianluca.baldassarre}@istc.cnr.it ‡Manuscript received ; revised .